Questions and Answers about Ten Formal Methods

B description speci cally helps to identify the nature of the service o ered to customers from the customer viewpoint. Z animation has made it possible to answer the following questions about the domain: The same service cannot be provided twice. If there is an authority for providing a service then it will be provided. QNAP2 and SWAP suggest di erent ways of con guring systems e.g. types of scheduling, multi-server options. The gives insight into the potential sources of bottlenecks in a real system. Any model will be an abstraction of reality; our model is particularly abstract. A model will thus leave unspeci ed various aspects. One needs to gain insight in such aspects before for example moving on to building a more detailed model. Question 12 Did the technique help you to identify missing parts of the model? epi Here are two examples of relevant questions, which have been explored using the epi description and the tools: What happens if two customers turn up simultaneously at two separate foreign o ces? Is it possible for a customer to get two o ces designated as her home o ce? latos animation has identi ed two potential problems with the epi version of the model, on which the latos version is based: Hidden return channels are not used throughout, instead some of the return channels are public. Using hidden return channels would make the speci cation more robust. The description should have been encapsulated (using the restriction operator to hide all free names). This would prevent public channels from interacting with rogue agents. Prolog has helped us to uncover one missing feature of the initial speci cation: The model is liable to a deadlock in the situation when o ce 1 asks the centre for information which resides at o ce 2 and vice versa. SuperVISE helped to discover the following problem: In an earlier version of the system, we had concentrated so much on authorising services at a foreign o ce that we had forgotten to revoke authorisation at the home o ce. This was brought to light by the simulation activity. Product Nets,QNAP2 No missing parts were identied since the model was not intended to be detailed or even complete. In general, omissions, if any, would have been found. Spin,Mur both detected immediately that, in the original description of the solution, if the centre does not know a customer, deadlock occurs. B The question that came to light whilst working on the B description was when the relevant parts of the data base are distributed. Z animation identi ed a situation where the state of a component was partially unde ned. A model might contain inconsistencies, or there may be discrepancies between an abstract and a more detailed version of a model. Such inconsistencies are harmful and need to be discovered. Question 13 Did the technique help to identify inconsistencies in descriptions? epi,latos ,Prolog ,Mur ,B did not help to discover inconsistencies. SuperVISE has the capability to nd inconsistencies in the model, but relies on the user to drive the simulations to recognise them should they occur. Product Nets The rst, abstract model contained a deliberate inconsistency in the sense that a customer disappears from the system after having been served. This inconsistency showed up after all customers in the system had been served. Spin did not help to nd inconsistencies, but one could imagine two o ces having the records of a single customer. In this case it would be necessary to verify that the customers get served only once. Z did not help to nd inconsistencies in any of the three models. We have not tried to prove properties of the models, but if we had, any inconsistencies would have come to light. QNAP2 has not helped us to discover inconsistencies but the SWAP tool checks that system-level con gurations are valid with respect to the parameters of the library components. 10 It was not suprising that more or less no inconsistencies were found since all the models are based on a single sample model. 4.2 Background Here we present information about the background of the experiments, such as the source of information used whilst building the models, and the motivation for building the model. Question 14 What is your description based on? epi model is based on recollection of oral presentations and documents on the real system. latos model is closely based on the epi description. Prolog ,SuperVISE,B,Z,QNAP2 The description is based on recollection of oral presentation. Product Nets The rst, abstract model is based on oral presentation and documentation. The second more detailed model is based on independent development. Spin model was based on recollection of an oral presentation. This turned out to be not entirely correct. Interestingly, the experimentation with Spin brought the misunderstanding to light. The incorrect interpretation assumed that the centre would sometimes communicate directly with the customer. However, a customer only communicates with an o ce, which then communicates with the centre on the customers behalf. Mur model is based on recollection of oral presentation and the Spin model. The methods that we have used are mostly either process based or state based, but we have also used some other approaches. It is interesting to see that more than anything else, the nature of the method determines the focus of the model, and also the way the components of the system are modelled. Question 15 What is the main focus of the description? epi model focuses on message passing. To describe this it was necessary to also provide a description of the internal state of the processes involved. latos The centre, o ces and customers are processes, communicating over channels with xed names. All other objects are represented by names passed over the channels. Prolog The centre, o ces and customers are processes, communicating over channels with xed names. The data bases are facts in the Prolog system. SuperVISE focuses on concurrent processes exchanging messages. The data bases are represented as local state owned by the processes. Product Nets The focus of the model is provided by the customers. They are represented as tokens wandering through the network. O ces are also represented as tokens, but they remain at their positions. The centre is represented by a sub network. It can be viewed as an active component. Communication is represented by ring of transitions. The data base as maintained by the centre has the form of a multi set. Spin model focuses on the processes and communications. The data base was modelled as a pair of arrays. Mur is mainly state based. Therefore, modelling in mur focuses on breaking the life cycle of a process in phases. This is necessary to make explicit the points at which interaction between sections of code occurs and points at which non-deterministic choice occurs. In addition, one must ensure that an explicit handshake takes place for each synchronous communication. B The service is represented as an abstract state machine with which users interact. Both the description and design are described as single state machines. The components are modelled implicitly by the operations and state variables that relate to them, e.g., the centre is modelled implicitly by the operations for distributing services and for dealing with non-local queries. The data is represented by appropriately typed state variables. Communication is modelled by operations. Z The focus of the level 0 and 1 models is state, the focus of the more detailed model is state combined with communication. In all models the data bases are modelled by partial functions, customers are not explicitly modelled. The centre and the o ces are modelled depending on the level of detail. In the most abstract model no centre or o ces exist. At level 1 the centre and the o ces are modelled by state, and at level 2 they are modelled by more detailed state and the ability to send and receive messages. At level 2 communications is modelled by a relation. QNAP2 model focuses on the performance characteristics of a queueing network. The most important characteristics are the workload imposed by the customers, and the quality of service obtained by the customers. 11 Tool support is important, as studying a model involves many repetitive tasks that can successfully be automated. The user is then encouraged to concentrate on the intellectual challenges in the experiment. Question 16 What tools did you use and why? epi animator and associated state space search engine proved invaluable to explore the full state space. latos translates the model, as well as the description of the operational semantics of the monadic -calculus into a Miranda [38] program. This program explores the entire state space. Prolog Sicstus Prolog [6] has been used for the experiments. The logen partial evaluator [19] has been used to compile the -calculus semantics with the model into low level Prolog so as to allow for faster animation. SuperVISE is the translator from VHDL+ to VHDL, and ModelSim is the VHDL simulator. Product Nets The tools use are the Product net machine and the SH veri cation tool. They o er: a graphical editor to create the speci cation, a project administration tool to manage speci cations consisting of multiple subnets, a simulation tool on the graphical level, a textual simulator with di erent operation modes (user driven/ randomly driven, stepwise simulation/ multi-step simulation), an exhaustive simulator (complete reachability analyser), an abstraction tool to decrease state-space sizes and to perform certain consistency checks, a temporal logic model checker including linear and approximate satisfaction (i.e., satisfaction under fairness) of properties. The di erent parts of the tool are integrated by a common user interface that allows for a unique access to the di erent parts of the system. Spin provides an integrated environment (Xspin) running under X-windows with a simulator, veri er, message animator, LTL manager, and nite state machine viewer of control graphs. The tools are nicely cross referenced, so that clicking on an object or event in one window highlights corresponding parts of other windows. The Spin simulator can be driven interactively or by traces provided by the veri er. Spin was chose because of its maturity, the fact that it is widely available and free. Mur has a Spartan interface consisting of the Mur compiler (which generates C++). B toolkit was used to syntax-check and animate the description. It was also used to check the consistency of the description. Z Three tools have been used to support the modelling in Z: The LATEX document processing with appropriate styles for producing documents and slides. The Z/EVES system for type checking of the LATEX source. Z/EVES also supports interactive theorem proving but we have not made use of this facility. The PiZA system animates the speci cation. Its input is a direct translation of the LATEX sources. The ability to work with one single source is an advantage. QNAP2 is an established tool for calculating properties of queueing networks. A possible alternative would be to use a general purpose tool such as Mathematica. The Swap tool was used because of our involvement in its development. Not all experiments have been performed with the same motivation. For example one might be inspired by the problem itself, or one might be interested in applying a particular tool. We do not believe that the motivation has actually in uenced the outcomes because whatever the motivation, the experiment wants to be successful. Here success is measured in terms of the number of discoveries, either about the problem domain, the model or the method. Question 17 Why did you write the description? epi description was created to understand the principles of the real service. latos was used to study an application of the -calculus semantics. Prolog and the ecce partial evaluation system [22] has been used to experiment with partial evaluation and abstract interpretation as a form of model checking. This is a fairly recent and promising idea [20, 21], which we are planning to pursue further. 12 SuperVISE was used to evaluate the tools for the purpose of modelling distributed systems. Product Nets The Product net model was created to be able to compare the tools and the technique with the other tools. Spin,Mur were used to show that model checking is a valid approach to studying distributed systems, and that model checking can provide useful insights. B was used to understand and clarify the problem domain and to produce a provably-correct design. Z models were created to help understand the problem and to show that Z is useful for describing models of distributed systems. QNAP2 was used to show that even small performance models might be useful. Swap was used because we wished to exercise it. 5 Conclusion 5.1 Considering Each Method Separately The nal question attempts to summarise the particular experiences. We should like to stress that our conclusions apply to the particular models we built. Our conclusions do not necessarily carry over to other modelling activities. Question 18 What is the main conclusion about using your technique/tool? epi In a sense the epi model falls in between two stools. It is not easy enough to use for real programmers. The need for real data structures to model the internal state of the processes makes the model unnecessarily complicated from the point of view of the theory. latos is of limited use in its present state because the tool is not user friendly. Prolog Prolog is a good tool to implement speci cation languages, and to experiment with speci cations written in such languages. The promise of nite state and in nite state model checking provides a further incentive to use Prolog with partial evaluation. SuperVISE in its present form is not the most appropriate tool. It requires a user to: write VHDL-like code which is unfriendly and unnatural to software engineers. use a VHDL simulator However, SuperVISE has some interesting and powerful features which have not been exposed here, notably multi-level modelling and the power of the SuperVISE interface which enables the execution of models assembled from components described in differing levels of detail. Product Nets are useful for the speci cation of systems at the architectural level, to compare di erent designs, to search for errors in high level designs. A typical speci cation would be far removed from an actual implementation and currently it is not possible to automatically generate code. Spin,Mur Spin is more powerful than Mur . Its notation is appropriate for modelling concurrent systems, even though the data structuring facilities are primitive. Mur would be useful as an alternative if Spin were not available, although it is unable to deal with liveness properties. B is appropriate to specify a distributed database problem. Some training is required but once the method is understood, the tools are easy to use. Z Our experience with modelling in Z shows that the abstract description is clear, concise, and useful. The most detailed description is perhaps not best done in Z because of the lack of support for concurrency in Z. The tools were found to be surprisingly easy to use, because they interwork well. QNAP2 and Swap have a sound theoretical basis in queueing theory. The method is well established and of practical value. The Swap tool is currently under development. The user of the method/tools has to be skilled in the interpretation of the statistical results. We believe that the design of any distributed system should be guided by its performance characteristics. It is important to keep the models up-to-date after a system has been delivered, as hardware and software con gurations will change, thus a ecting the performance characteristics. Table 1 summarises the main ndings, which indicate that: The -calculus gives the most elegant (i.e. concise) model of the application. Product Nets are more verbose but easier to understand because they are expressed graphically. Model checkers based on linear-time temporal logic (Product Nets and Spin) give the most comprehensive information about the model. The integrated tools (B-tool, Spin and Product nets) provide the best support. 13 . Process based State based #.Question epi latos Prolog SuperVISE Product-nets Spin Mur